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ABSTRACT 

Arguing that user studies and performance measures 
provide two major approaches to evaluating the effectiveness of 
library services, this occasional paper presents an overview of the 
literature, examines the goals and approaches in iser studies and 
performance measures, and provides suggestions for increasing the 
potential benefits of both by combining the two ^ ^chniques. The 
resulting approach — performance measures based on user studies — is 
then discussed in terms of data collection, possible benefits, and 
limitations. It is concluded that: (1) valid procedures, i.e., 
measures that actually measure what they purport to measure, must be 
utilized by libraries to adequately evaluate their services; (2) 
libraries should be most concerned with measuring their ultimate 
product, performance or effectiveness, based on user data such as 
satisfaction; (3) a real need exists for libraries to be accountable 
for the effectiveness of their services; and (4) user^oriented 
performance measures provide a valid evaluation technique. (A 
181-item reference list concludes the document.) (CGD) 
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IN TRODl CTION 



In 1980. Mai V Jo l.> n( h, (lire( lor of (he Offu e for Restan h of ihc Amci k an 
Libran Association, wrote: "For at l^^ast 10 years the publu libiar\ (om- 
muniiy has been struggling to find practical methods of evaluating the 
effectiveness of public libraries."^ But as Beeler stated in his work on the 
measuremeni of library service, "There is probabU no measurement task 
which public ser\ants face which is more diffn ult than that of measuring 
the quality of service."^ 

Within recent years, efforts to determine the qualit\ of libran services 
and or their effectiveness ha\e taken at least two major approa( hes. One 
approach has focused on usei studies. According to D'Elia, "public librar- 
ians have recognized that the planning and evaluation of library services 
must be predicated upon an understanding of the behavior of tne librar\ 's 
constituencies."'^ 

A second important approach to measui ing the quality of librar> serMces 
has concentrated on measuring library performance. Efforts in this area 
were given a major boost in the 1970s by the work of DeProspo, Altnian, 
and Beasley^and there has continued to be considerable support within the 
library profession for the consideration of performance. In 1982 the Ameri- 
can Library Association published its Output Measures jor Public Librar- 
ies^ dnd a second edition was published in June 1987. (Mary Jo Lynch, ma 
1983 article,^ discussed the relationship between DePiospo's Performance 
Measures for Public Libraries and Output Measures for Public Lihraties,) 
In 1984, the Asscxiation of Research Libraries published a manual of 
performance measures for ac ademic and research libraries, ^and in 1985, an 
occ asional paper focused on performance measurement for public ser\ ic es 
in ac ademic and research libraries.® The Committee on Performance Mea- 
sures of the Association for College and Research Libraries is currently 
de\eloping plans for a perfc)rm:»nce measuies manual for academic 
libraries. 

At least one crucial question remains, however. What, if any, is or should 
be the relationship between user studies and peiformance measures? A 
re\iew of the literatUiC suggests that one important source of performance 
data is the library user. In other words, the performance of a library, 
measured in terms of how well f is meeting the needs of its users (and 
nonusers), is one of the most meaningful ways of judging thequalit> and 
effectiveness of a library's services. As Burns stated: "Lsers are essential to 
the ba'^ic mission of libraries and are the only data sets that contain both 
input [resource] and output [jXTformance] measures of system ac ti\ ity."^ 
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Therefore, whar (oHous are ()\er\ie\Ns of user siiidie:* and fx^rformaiKe 
measures and suggestions for ma\niii/ing the potential benefits of both by 
(ombining the two techniques. 



rSKR STl'DIES 

"The hterature of user studu s is large and varied. It ranges in complexity 
from detailed reseaich in\ est igat ions, which model how a user gathers 
infoimation, to the most elementary, in house, descripti\e studies of a 
single librar\ ."^^ One of the best known, and still most important, of the 
national user studies was conducted bv Qimpbell and Met/ner and was 
published in 1950." In 1978 the Gallup organization reported the resultsof 
a national sur\e> of library users^^ and it has conducted another survey of 
libr;tr> use more recently. In addition to the various national studies, many 
user studies have hern conducted for states, regions, i nd local 
communities. 

Ideniifuation of the many user studies has been aided by the availability of 
se\eral bibliographies. Among them are publications by Albright, Atkin, 
Bales, Davis and BaFle> , Ford, The International Federation foi Documen- 
tation, Lubans, Slater, and Wood." In addition, the Annual Review of 
hiformation Science and Technology has included a summary andbiblio- 
graph> of the previous >ear's activities relating to user studies. A useful 
siimmar> of the findings of several major user studies was published by 
/wei/ig and Dervin in the 1977 \ ol'ime of Advances in Librarianship}^ 

Further confirmation of the growing interest in library user studies was 
provided b> I^incaster in his book on the measurement and evaluation of 
librar\ services. He |)ointed out that library surveys are shifting their 
emphasis toward the library user, patterns of library use, and the degree to 
w hi( h user needs are Ix ing met In an Occasional Papers published by the 
rniversit> of Illinois in 1983, Clark identified new approaches to the 
measurement of public *.it>rary use and presented a "model for public 
librarians who wish to study the patterns of use b\ induiduals in their 
libraries "^^ 

Evidence that libraries otiier than public libraries are bee coming increas- 
ingly interested in their u>ers was prcnided by ^76 and 1981 SPEC Kits 
published by the Association of Research Libra. i^s.^^ Both publications 
ueiv devoted to user studies of uni\ersit> libraries, and the 1976 kit noted 
that a lairlv large per^enta<^e of user surveys had been employed, in part, 
**lo"\aluate libraiy se» vices in terms of user response to those services. "^^Ii» 
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addilion, ihnv ha\e Ixrn sf\nal artidts published wiihin iIk' last few 
>earsdis(iissingar»demi( libran user siudies.^^N'or are spec lal libraries an 
ex( eplion to this irend. The pre\ lousK mentioned summar\ of user s(U(hes 
appearing in ihc Annual Review of Information Science and Technology 
is gcneralK (oncerned wiih the informaiion needs and uses of sdcntisls 
and (ethnologists in special libraiv sellings 

rnforiunateh, as Line has noled. "The* leraiureori user needs' has been 
confused b> imprecise use of lerms. "^^ Noi onl\ have \arious auihors 
provided, or ai leasi implied, different meanings for ihe same lerms, bui 
some wriicrs have lended lo use, inierchangeably, lerms thai in fad have 
differeni meanings. Two of ihe lerms most commonly confused have been 
"use" and "user." Zweizig, in a 1977 ariicle, ouilined ihree conceptual 
approaches to the measurement of library use.^^ The first concept he 
defined as u^e-— i.e., internal library activities such as circulation. The 
secoiid concept he id ntified as the user, representing a shift in fccus from 
the library activity to the library patron. He defined the third concept as 
uses, suggesting a shift in emphasis from the patron to theexternal use that 
the patron makes of the library's resources. In the same article, Zweizig 
pointed out that there had been relatively few studies that had restricted 
their attention to the actual user, as opposed to considering use as well.^^ 
D'l'llia, in an artic le published in 1980, also shed some light on the 
distmc tion between user studies and utc studies.^^ He defined user studies 
as those studies concerned with the characteristics of tisers (and nonusers). 
He defined use studies as in\estigations of the nature and extent of the 
library materials and services used by patrons. 

At this point. It appears reasonable to define libi .y use as those ac tivities 
which occur primarily within the libiar\ and which reflect rather tradi- 
tional library functions *uch as circulating books and ansv/ering reference 
c^uestions. According to D Elia, it might be useful to categorize the basic 
types of use as (1) frecjuency of use, (2) intensit\ of use, and (3) in-house 
use. And it would no doubt be profitable to heed the advice of Dervin, 
vvho suggested that librarians c onsider rhe t> pes of siruations that result :n 
people UMng libraries and their resources.^^ 

However, this paper is more concerned with user studies than with use 
studies. As Ford has written' "Of more use [than library use surveys] are the 
studies of people'** informaiion needs and information seeking behavior, 
panic ularly where these are based on whatactualU happens rather than on 
people's opinions of what might happen. hi other words, "user" studies 
should foe us ncjt on what libraries do. but on what people do, or wish they 
could do if the> could obtain the necessary information/^' It is important 



"(hat user studies distinguish between an e\aluati()n oi measurement of 
the su(ress of a ser\ ice and an\ anaU^is or measuiement of the usei of tliai 
ser\i(e In attempting to phue user studies in perspe( ti\e, one might 
fonsider the scheme outlined b\ Ford. He referred to. (1) information 
transfer, (2) information need, (3) information use, (1) information rele- 

29 

vance, and (3) information users It is this fifth (omponent that most 
conrerns here. 

l^ser studies generally ha\e fallen into one of two categories — in-house 
surveys of users and (ommunity analyses, which usually (onsider both 
library users and nonusers. While the emphasis here is on the latter type of 
"user** studv, there is considerable support within the librar> profession 
for studies which concentrate on actual users hi 1 98l, Thomas Ballard was 
arguing in American Libraries that public libraries should focus on users, 
not nonusers He contended that: "When suggest ioris for impro\ement 
ha\e been offered [by nonusers], they are not the innovative suggestions 
hbrarians seek but ratht r more of traditional library services — more books, 
loi ger ho u IS, or better parking. **^° Others have argued for many years t!i at 
public libraries, with their limited budgets, cannot possibly serve well all 
members of their communities, so they should concentrau* their resources 
on current users. 

The purpose of in-house surveys of users received attention in the Publi' 
Library Association *s Plamung Process for Public Libraries. Its authors 
stated: "Tlie primary purposeof an in -library or user survey is to determine 
who uses the library, how much, and for what, and to ask users about their 
attitudes toward and perceotions of the library. **^^ Burns, in effect, catego- 
rized this information as demographic data, pieferential data (e g., reading 
preferences), and behavioral data (time and length of library visit, etc ) 
He also proposed several measures of user satisfaction, including: the 
proportion of "hits,** the users* perception of the library, ( ir(ulations per 
type of user, success in ha\'ing reference cjuestions answered, and document 
deliN'ery time.^^ A Platining P' ocess for Public Libraries suggested (ollect- 
ing user data on. characteristics of the users, their purposes for using th^' 
library, services used and subsequent level of satisfaction, reasons for any 
dissatisfaction, materials used and their availability , users* searc h patterns, 
additional library services needed as percened by users, and priorities 
assigned to ser\ iccs by users. According to A Planning Process (or Public 
Libraries, the information generally deri\ed from a community analysis 
tends to provide two basic profiles — one of the library *s environment and 
one of the library's population— and this information can bequuediverse 



rn iiaiiiif. Foi tAample, sonic oi iht basu i\ pt sol data tliat (an bt'obiaiiud 
from a (iii/tn sii»\(\ indiuk* 

1. aitiliides low aid ihv piiblu libian and its lok*. 

2. indi\ idiials' saiisfadion wiih librar\ sn\i(t'>, 

3. information irgaidini; n on users, 

4. ukntification of libian materials used, and 

f). deniographu ( hara( ttristus of users and nonust rs 

Additional information poteritialh gatherable nu lu(k s 

1. riti/ens' perct ptions of their information needs, 

2. riti/ens* per( eptions and altitudes regarding their i)ubli( librarx, 

3. their awareness of library services, 

4. (iti/ens* evaluations of their access to the hbrar\, 
3. their perceived reasons for not using their hbrarv, 

6. ahernati\e sources of information, 

7. reactions to librar> |X)li( y changes, and 

8. the geographical locations of both users and nonusers.^^ 

Ford, in his work on user studies, identified **s\ stems" to whic h users tend 
to belong. He labeled these as cultural s\ stems, political systems, member- 
ship groups, reference groups, invisible colleges, formal organizations, 
project teams, the indi\idual, legal economic systems, and information 
markelplac es.^ Some of the best known community analyses weredirected 
by Lowell Martrn.^ Other analyses reported in the literature include 
studies conducted by Ciiirpenter, Chen, and Joyce.^^ 

Data Collection 

Both types of user studies disc ussed thus far ha\ e employ ed a variety of data 
collec tion lechnicjues and tools, but "the cjuestionnaire and interview are 
still predominant..."^^ On the other hand, newer technicjues, such as 
modeling, are beginning to have some impac t on the des\gn of user studies. 
DElia, for example, maintained the importanceof a priori model building 
in user behavior researc h He pointed out »he need for an understanding 
of the complex beha\ lors associated with use. He also staled that models 
might suggest possible courses of action that a library could take to try to 
influence user bc^havior. 

Additional techniques applicable to user studies ha\e been identified by 
Burns as including the RAM de\ ice, field studies nuolving direct observa- 
tion, the critical incidem technique, aiid citation c ounting.^^ Ford pro- 
post^ measuring ihe use of documents b> collecting data on loans, 
recjuests, citation analyses, solution records (a tvpe of diary), social and 




dfiiiograpliK t hai.H it*iisii( s, past u stMU li. hoidings, diai it s. and ( ontfiii 
anal\srs. In addition lo it roids rdaiing (o do( iiiiK'nis, hv suggt strd utiliz- 
ing obsmaiion. qiifstionnaiu's, and inttr\K*\Ns to (()Hf( t data/^ 

I^uu aster noted that the \aiious hbrau siirvev techniques ha\e included 
the utilization of diarits. operations reseaich. questionnaires. inter\ie\vs. 
do( umentaix anahsis, (hecklists, e\alaation \isits by experts, staustics, 
records, and standards/^ Basically two t\pes of methods were put forth b> 
PLA's A Plamiing Process for Public Libraries; the> ni\ol\e the use of 
interviews (parti( ularlv exit inteuiews) and cjuestionnaires (either self- 
administered or distributed and collected by librar\ staff membeis). 

Regardless of the spec ific tec hniques or tCK)ls emplo\ecl. user studies have 
tended to leflect certain basic assumpuons Buins's assumptions included 
the following. 

1 Psers and then reactions to the library are the ke\ to high quality 
seiMce. 

2. A usei stud\ shoukhonsiderusersboth in theaggregateasstatisticsand 
m the particular as indi\iduals. 

3. A user stud\ should coiisidei both users and nonuseis. 

4. A user study should be an ongoing process. 

5 The panic ular instruments or tools used shoulu desc ribe user response 
in a\aiiet> of wa\ s m one or more formats (desc ripii\ e nai rati ves, scaltrd 
responses, etc.) 

6. A detailed anahsis of the libiarv's comnuinit> or environment is 
essential. 

7. It IS particulail\ difficult to delcTmine causal lelationships 

8. It IS feasible to measure and cjuaniify the impact of libiaiies on the 
educational piocess. 

9. An indication of cjuali'\ CcUi be cleri\ed fiom cjuantitaiive 
measurement.'*^ 

Possible Uenefits 

But what of the goals and objec li\es of usei studies? What aie the benefits 
that librarians hope to reali/e^ Quoting Burns once again. 

riic goal of Use User siiiclics is the dis(()\('i\, arnt ulanoii, unde^siand- 
iiii;, niflueiuing, and. when appiopriaf, the chminaiion or ai Icasi 
iiiinimi/aiion of t!u)sc ohsuulcs bciwcfii a usei and liis information 
goals riu'sc chsiac Ics aie found in ihv so( i.d. insuiunonal. geogr.iphi- 
cal, temporal, and org :iii/atiC)n.d oi picxedur.d spa((' scparaiinga user 
fiom the Item oi infornKUion that wdl satisfy ins need 



8 




ERLC 



Or as stated succinctly by Zweizig and Den in: "User-oriented program 
planning is required to provide more responsive, accountable service. But 
user-oriented program planning requires understanding of the sub- 
populations to be seived."^^ 

Ford argued that the major aim of user studies is to assist in the design and 
improvement of in formation systems and proceeded to identify the follow- 
ing specific objectives and benefits: 

1. greater understanding of the proce ses of information tiansfer; 

2. improvement of information transfer and the organization of 
communication; 

3. modification of circulation services; 

4. more information about print readability; 

5. determination of the relationship between user performance and var- 
ious types of catalogs, etc.; 

6. more information regarding users' work habits; and 

7. more awareness of the possible applications of user information to 
administrative pioblems and decisions relating to ^ udgeting, staffing, 
etc.^® 

Busha and Harter, in their text on research methods, noted that user studies 
are needed to justify and expand library services and usage and io learn 
more about how people communicate. More specifically, they stated that 
user studies are needed to: predict library usage; determine why people do 
or do not use libiaries; identify what groups borrow which kinds of 
materials; identify what groups use which services; suggest how use can be 
encouraged; explore how urban, suburban, and rural use patterns differ; 
measure the effects of mass med'a on library use; and identify actual 
needs.'^^ A Planning Process for Public Libraries indicated that user sur- 
veys can provide information about the proportion of the total population 
using the library, the proportions of population subgroups using the 
library, user nonuser awareness of services, the levels of and reasons for 
user dissatisfaction, immet needs, types of materials used, and the reasons 
why individuals use various resources.^® And last, but not least, as implied 
by some of the many benefits just itemized, user studies can measure, at 
least to some extent, user satisfaction with existing library services. Tiie 
potential importance of this kind of information was emphasized by 
Kantor in a 1976 article in which he argued that one of the strongest 
indicators of ihe transmission and growth of knowledge is the library 
users' judgment or satisfaction.^^ Experts do not agree, however, on the 
validity of user satisfaction as a criterion for measuring library peifor- 
mance, and this issue will be discussed further later. 



While the liM of poiviuial Ixnefiis 's no douhi ^^u^^ln than ihv actual 
benefits ganifd thus fai fiom usei studies lihiaiians and otheis ha\ emacie 
substantial additions to then ()\vled,^e of libtarv usns, Soiiie of this 
infoimation might \vell be (onsideied to be hu to wuious local 
situafions, other information ^eeiiis to lepje lei bioad piincples 

Ford, for example, summaii/ed some liasr ]iiitUi])les ol usei lx*ha\ior as 
follows: 

1 INeis of information Ix'lon^i^ u) identifiable gioup^ with : haiac teristu 

patterns of infoimation requiienients. 
2. The U)k* of the u**ei is an iinpoitant deteiminani of his or he r inU)inia* 

tion need, 

S. AccessibihtN is a ke\ fat tor affec tin^ the use of an information source. 
1, l^ser awareness of. and al)ihc\ to use, infoimation sources is often 
impel feet. 

5 hiterpersoiK communicaticjii is one of the most impoitant mean^ of 
transmitting information 

6 The amount of information iec]uireci \aiies tonsidt My among 
individuals. 

7. lasers often require mloimaron to be supplied on shoit notice. re:r ud- 
less cif the a\ailabilit\ of such information 

On the other hand, there is stili muc h that we do not know tibout librarv 
users and their infoimation seeking Ix^havioi. Usei studies ha\e not yet 
answered all of oui questions. Most of the lesearch voncerning users has 
tended to be descripine in r and has not measuied adecjuately con- 
cepts sue h asuser satisfac tioF ii.c f "h isc lear that, with the exception of 
education, demc^graphic \ ariables ha\'e prcnen of little \alue in predic ting 
whv adults use librai ies.**^^ Zwei/ig and Dei \ in also pointed out the 
inadequacy of purelv desc ripti\ e measuies in stating that* "ThenumlK*rof 
users in the library is a measure of lil)rai\ acti\ it\, but it is questionable 
whether it is a measuie of libiar\ effec ti\eness hi Vltman's book on 
public library admin isti a tion, Zwei/ig pointed out that* "Commurntx 
analysis w ill not result in direc t identihc ation of c omnuinii\ information 

,' ,.53 

needs. 
Limitanons 

What are the problems that, thus fai at least, have lesulted in the inability 
of user studies to full\ measure libi^rN effec tneness^ No doubt theie aie 
se\'eral, but some of the concerns raise by Burns .nv worth consideiing* 

1. It is unlikely that contemporary public libraiiescan meet adequately all 
of the demands being placed U| )n them, but how to measuie this 
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shortfall and how lo <.kit*imiiu' an a((q)ial)k* \i\v\ ' toiifall au' 
diffiruli pioblems *o resoht*. 

2. Diffm*ni (lasses of iiseis tend to pla(f difltunt demands on ihv piihlK 
lihiary. 

3. A small peut ntageof i» libian s potential ( lu nteleat ( oiints for most of 
Its use. 

4. The degree of similarity between information -seeking behavioi and 
their relationship to the piocess of (ommiiniration ha\en()t lx*en fully 
determined 

5. Further exploraMon of the effect of the pinuiple of least effoit on the 
user and his her information-gathering behaMor is needed 

6. More research is iieededon institutional diffeieiK esand similarities and 
their effect on user activities and behavior. 

Other potential limitations of user studies have Ixrn identified in the 
literature as well. The DuMonts pointed out the nnportance of measuring 
the impact of a (hanging environment in that the library must "interact 
with individuals and its communities in a vaiiety of wa>s over time. 
White noted that "user studies that simply ask patrons what they warn or 
how well the\ like what has been provided evoke on^ a self-fulfilling 
prophecy. People state an expec tation for what they have gotten in the p<<st 
and for what they think is reasonable toexpe( t in the future. "^^ Ford stated 
that: "There has been a strong tenden(\ in all user studies to equate u^e 
with value. There are a number of objections to this ..."^^ Wilson com- 
mented that one of the most r.egle( ted areas is the study of information use 
and exchange.^ 

Zwei/ig pointed out that user studies have been limited b\ the fa( t that they 
have not measured the library's a( tual contribution — i.e., how the patron 
uses the library's resources and services and what then \'alue is to him or 
her.^^ Similarly, H arris and Sodt concluded that traditional user studies 
have ac c omulished about as muc h as thev c an and we need, foi example, to 
try to determine the \alue users derive fic^m lil)iar\ use.^^ 



In light of the apparent limitations of user studies in e\aluating the real 
effec liveness of public libraries, some librarians in rec ent years ha\'e turned 
to other evaluatixe tc hniques. hi fac t. Knightly identified five c lasses of 
evaluation — (1) effort evaluation (inputs), (2) process evaluation, (3) effec- 
tiveness evaluation (outputs), (4) impact evaluati ""id (5) cost- 
effecti\eness and cost-benefit.^^ The third mentu ned category, 
effecti\cness evaluation, has received considerable attention and has often 
lx'*n referred to as performance measurement or output measurement. 
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But what f\cUil\ IS nu*ar.» In priloimaiKe measiiiemeni oi nu'asuus and 
how do ihv) difftr fioni oilu'i t\p^*s()f t*\aliiali()ii? In a \viivi lo Robeil 
Bums, Manann k Bioun pointed to one important distuution between 
measures of jx'iformance or outcomes and measuies of a(ti\it\: "A( liviix 
measures nidicate the le\e] oi amount of various kinds of a( livities within 
the hbran (e.g., (ounts of the numbeis of uems pioduced, of the numbers 
of refeience questions received, numbers of pations ser\ed, et(.). On the 
other hand, oUKome measuie* seive to indicate what was aaomplished 
(what puipose oi obje( ti\es wereac hie\ed)asii result of this program maiK 

.,62 

a( tiMty. 

Howe\cr, as has Ix'en pointed out b\ Philip Morgan* "Output (an only be 
arhiexed h) some input of lesourre*^, and he defines input as "the 
volume of resources of labour, Kmd, t;me, fninnte, etc. that contribute to 
(he a( hie\tmeni of outputs Ye!, "pcifoimance measure^ focus on nidi- 
(ators of librar\ effectiveness and output rather than input alone, and are 
( losely related to the impact of the library on (he ( ommuni(\ ."^^ Or as was 
similar I v stated in A Planmug Process for Publu Libraries: "Performance 
measures are distinguished from libraiy statistics in that (he former focus 
on librai\ effec ti\eness, that is, adec]uac\ of performance, and on the 
impact of the librar\ on its community "^^ Flsew here, output measure- 
ments have been defined as indicators of* 

1, the degiee to w hic h an organization meets the needs of its community, 

2, the extent to which an organi/alion achieves its c^bjectives, 

3, the effectiveness of an organization, and 

4, the impact of the activities of an organization on its community,^' 

Schiader defined performance measures as a i\pe of consumer or market 
leseaich with the emphasis on performance for the user.^^Blasingameand 
Lvnch indicated that output simply lepresenis what the user gets from a 
libiar\.^^ Hambuig and others stated that* " The real outputs of library 
seiMce aie the stimulated student or teacher, the scientific discovery, the 
infoimed \c)tei, the successful businessman, etc."'^^ Others have pointed 
out that the concept of "need" is implicit in performance measurement 
and have emphasized the imf)c)rtance of evaluating hem well it is being 
met,'* Ibis concept laises the issue of usei satisfaction once again 

A term cjuile siindai ir concept to performance, as used by Orr, is good- 
fiess. lit (Onsideied goodness to ha\e two basic aspec ts whic h are repres- 
ented b> two cjuestions: "How good is the sei\ ice?" and "How much good 
does it do?"'^ Oi in othei woids, what aie the cjuality and value of the 
libraiy's sen ices? 





Previous Studies 



rnfortiinau'h, as noitd In Lowell Marirn 'Studies of how libraries are 
used, and with what su((ess, have been less freciuent than the who, what, 
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when, and wheie variety/' Parker wrote that: "Most of the substantive 
researc h has been ( onduc ted and the results reported .n the hteraturesnu e 
1968."'^ She went on to ( omnient that most of the research studies ha not 
lK*en ( umulatne — i.e., ihe\ had not built on previous siufiies. Rather, most 
of them had been projec t-oriented. Mu^ h of the reported resear( h indicated 
a tendency to treat the libra r\ as an isolated entity anu to ignore its 
relationship to the larger information (oinplex in terms of its ability to 
provide information and doc uments. 

On the other hand, there has been (onsiderable iuppoit for, andactivriy 
related to, perforrnanre measures w ithin the last several years. The Public 
Library Mission Statement.., emphasized the irnpoitai (e of measuring 
output, as well as input, but at the same time acknowledged that: "The 
sedal indicators to measure library output ha\e yet tj Ix* defined."'^ /I 
Planning Process for Public Libraries dUo ik\ oted ( (jn^^iderable attention 
to performance measures. 

But as was noted earlier, the major impetus for the growth of interest in 
performance measures was the work done by DeProspo, Altman, and 
Beasley resuh.ng in the publication oi Performance Aieasures for Public 
Libraries in 1973. With regard to the rationale for *ne research resulting in 
this work, DeProspo stated: "Few antecedent approaches [studies] exist 
which the public library can utili/e fruitfully \n developing innovative 
approaches to measuring the performance of the services it offers its 
public ."^^rie also noted that a surve\ of public librarians had revealed that 
many of them distrusted statistics as measures of effecti /eness, and that 
they wanted more data that were people or user-oriented. Or as Gerald 
Born stated in the introduction: "New mcasuri's recognizing the satisfa(- 
lion of the user and a more adecjuate evahiation of library service were 
neecU^d."'^'^ 

This need was recogni/ed by the authors who stated that the primary 
purpose of their study was "to develop meaningful rndicators of perfor- 
mance whic h could be used by library ad nunistra tors to assess theeffe( tive- 
ness of their operations.**'^ The three basic areas that they c onsidered were 
collection availability, ac ti\ity level of hbrar\ services, and c hara( tcristics 
and satisfaction of users. 

The research that resulted in the DeProspo publication was cc)ndu( ted at 
the Bureau of Lrbrary and Information Sc lenc r o{ Rutgers University and 
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wys siippoi ted 1)\ a gfaiii lioin ihv V S. Olfnt'ol Kdiuaiion Ilu* Rutgt is 
gioiip ( ondiK ted a UMtw ol the ulaied liifi.uiiif and an.il\/t'd tlu tlifii 
(iirniit hl)iai\ statistK.d upoiimg N\MtiiiN The ivsi of ihv piojfci (on- 
sistfd of: (1) ihv (k'\t'l()[)nKiil of piihlu libi an tlfcc ti\fiK'ss (iitnia, 
(2) iIk* desij;!! of a iiK*iliodolog\ foi ihv (olkdion of appiopiiau* ( iittria 
data, (3) ilit* (olkc iion of data in a gioiip of pilot Ntud\ lihraiifs, ( 1) tlu* 
('stablishninit of tnitati\f ranges oi pnforman(t'. (3) tlu* testing of tlu* 
criteria and methodolog\ in a national sample of public libiaiies, and 
(6) the preparation of a "piofile" for eadi of tlu* sample libraiies, 

Fonowing the initial Rutgers project, a peifoiniance measures stud\ was 
conducted b\ the North Suburban Lil)iar\ S\stemof Illinois This study 
used the same basic nu*thods of data t ollec tion as weie used b\ DeProsjx), 
but the gioup of hbianes paitu ipating in the Illinois piojec t included a 
greaiei pei rentage of small libiaiies. 

In 1971, the Ial)iar\ Research Centei of the TnueisitN of Illinois was 
funded by the Illinois State Libraiv foi the puipose ol testing tlu* data 
collection tec hnic]ues whic h had been de\ eloped at Rutgeisand whic hhad 
been incoipoiated into a performance measures manual — Perjortnance 
Measures jor Public Libraries: A Protedures Manual for the Colled ion and 
Tabulation ^ Data^^ 

The Illinois piojec t unolved a sample of 78 public libiaiies lepresenting 
major budget categoiies and geographical aieas within the state. The 
Library Researc h Centei conduc ted se\eial uoikshops for the purpose of 
insirut ting partic ipating libraiiansin the use of the perfoiniance measures 
manual Partic ipating hbianes carried out thiee-da\ peiformance e\alua- 
tions tabulated and summari/ed their data, and leturned then lesults to 
the Library Reseaich Center for fuithei ana 1\ sis. The centei suljsecjuently 
analyzed the data, prepaied pioliles for the librai les, and submitted a final 
reix>rt to the stale hbraiy descnbing its expeiiences with the project and 
evaluating the perfoi .nance nK*asuies nicuuial.^^ Otbei )l)sei\atic)ns based 
on the Illinois stud\ ueie leported 1)\ Goldhor in a 1978 article.*^ 

In the nu'antime, a levised xersion ol the Rutgeis peiformance nu*asuies 
manual was used 1)\ Kllen Altman in an e\aluation of the St Petersburg, 
Florida, pi blic hbianes (Aliinan latei published a i)eiloimance mea- 
sures manual which lepiesented a ie\ision ol the original Rutgeis 
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manual ) 

Duiing this same peiiod. the Buie.ui ol L]l)i<u\ and Information Science 
was funded 1)\ the New jeise\ State Librau foi the purpose of testing the 




u*lial)ili(\ of tlif mcasiiu'ment lediniqiu's doped foi ihv innidl man- 
ual, developing addihonal measures of lihian ser\i(eavailal)ilit\ ni New 
Jersey, implementing a piogiam lo tiain New Jerse\ libiaiv peisonnel in 
the use of pei formanc e ineasuies, and investigating the feasibility of in( or- 
porating the measures into a statewide statistical reix)rting system.^'' 

Since then, several public libraries have conducted perfoi mane e evalua- 
tions utilising, to some d(\gree, the technicjues piesented in the DeProspo 
and Altman manuals. Examples of such studies cited in the litenMure 
inc lude ones reported by Fan held, Gregory, McKen/ie, Crane, and 
Ramsden.^^ These evaluations, as a result of being based on the original 
DeProspo work, tended to c oncentrate on measuring mateiials availabil- 
ity , eirc Illation statistics, building usage, patiernsof refeience usage, fac ili- 
iies usage, public service personnel, and user satisfacticm As this listing 
indicates, DeProspo's measures tended to emphasize le\els of ac tivitv 
rather than the users themselves. On the other hand, DeProspo's manual 
did provide for col lectin <^ to some extent al! of the types of "user" data 
identified by Morgan. Thes? included type of user, type of use, frequenc y or 
intensit> of use, and quality of or satisfaction with service. 

But given the types of information that performanc e or output evaluaticms 
could measuie, these earlier stud es did not incorporate all of them i)y an> 
means Tor example, the authois of Planning Process for Public Librar- 
les stated that: "Three kinds of infoimation contribute to the evaluation of 
the library's current performance* the statistics that most libraries keep 
routinely; responses on the citi/en, user, staff, and student suiveys; and 
measuies of the library's achievements relative to specihc objectives or 
servic c*s, that is, performance mc asures."^^ The authors then spec ihed these 
measures ol performance as: 

1. cnerall citi/en satisfaction with the library's sei\ices, 
2 perceptions of nonusers u\garding the librarx, 

3. numbers of legistcwl boi rowers or users. 

4. seivice aiea penetration (propoition of cui/ens who use the lii)rar\), 
3. user satisfac tion, 

6. user service hours, 

7. le\el of use of facilities and mateiials, 

8. in-librar\ circulation statistics. 

9. circulation outside the library, 

10. availability of mateiials, 

11. time delays i' obtaining materials, 

12. reference service use, 

13 the aiirar tivcMiess and accessil)ilit> of facilities. 
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1 } s(<«if a\ailal)ilit\, mu\ 
15. staff attitudrs 



In (onliasi, Sdilukbiti Kkiitifud ihosr areas wIik h (an Ik* (•\i)1()K'J I)\ 
peifoiniaiK f nKasurt iiKiii as iik hiding staff ev aluation, spac v utili/atKMi, 
(ollation evaluation, growth projection*. nieasurenKnt of advertising 
impac t, utilization of stafi, effec tiveness of scheduled operating hours, and 
(osl-henefit anal \ sis (At a glance, Sc hlukbier \ ( ri terra appeal to be less 
user-oriented than do those presental in the PLA work.) Gra\son and 
Wingate, in an aitide on perlorniance measurement in libraries, argued 
that user satisfac tion is the ke\ to measuring the effec ti\eness of a librar> 
and contended that duulatron is the single best measure. They did note, 
howexer, that c irc ulation figures must be related to the population served 
in or^ler to be meaningful — that is, the best measure is the percentage of the 
poi)ulatic)n using the librar\.^^ 

An additional approach to performance measurement was espoused b> 
Hamburg, and others, who argued that the most meaningful t\pe of 
performance measure is user exix)sure to documents. Hamburg identified 
three i\pes of document e\ix)sure measures, exposure counts, item-use 
days, and exposure trme.^^ 

A 1975 publication of the I abrar\ Assoc latron provrded \et another c lassi- 
fication of output measures. Thrs work frrst defined depth, or breadth, of 
service as the rmpact of the librarv ser\ice rn the community, h then 
dixided thrs impact mto rntermecliate output (service currently pro\'ided) 
and final output (the effec tr\enessof the ser\ rc e, possrblv measured rn use), 
and finally, rt rdentrfred \arrous output measures, rncludrng: 

1. the number of users, 

2. communrty survey data (who is usrng the frbrar\ and wh\ c)rwh\ not, 
user awarene.s of Irbrary ser\rces, patron attitudes, etc ), 

S cjuantrtative anah srs of c rrc ulatron, 

4. qualrtatrve analysis of c rrc ulation, 

5. reference use, 

6. reader satrsfactron (materrals a\arlabrlrt\ , attrtucles toward staff and 
facrlrtres, retentron or egrsterecl borrowers), 

7. extramural activrtres (lectures, concerts, etc ), 

8. servrce to housebound readers, and 

9. staff output.^^ 

Kantor, rn his manual of performance measures for academrc and research 
libraries, ^ described three measures — avarlabrlrty of lrbrar\ maierrals, 
ac cessrbrlrty of Irbrary materrals, and delay analysrs of spec rfrc at ti\ itres. A 





fourth mcasuic, anaUsis of pat»on a(ti\jt\, was tested but not included in 
the manual. Zwei/ig and Rodger's Output Measures for Public Libraries^^ 
included twelve peiforniance measures identified as. (1) circulation per 
capita, (2) m-library materials use i)er capita, (3) library visits per capita, 
^4) program attendance per capita, (3) reference transactions per capita, 
(6) reference fill rate, (7) title fill rate, (8) subject and author fill rate, 
(9) browsers' fill rate, (10) registration as a percentage of ix)pulation, 
(11) turnovcT rate, and (12) document delnerv. 

Data Collection 

A variety of technicines and tools ha\e been employed to measure the 
\arious performance indicators. Among the methods of measurement 
discussed in the literature is the collection of statistics. Statistics have 
been used to compare past and present use of a librar\, to compare one 
library's situation with another's, and to measure performance w ith regard 
to predetermines! standards. 

Other studies have utili/ed questionnaires and or interviews These tools 
aie particularly useful for measuring final output or impact and are often 
used in conjunction with a community analysis to assess the impact of 
partic ular librar\ services. A few studies have used formulas or models that 
can consider the variables that influence performance. 

The DeProspo study employed a ' user ticket" which was in effect a short 
questionnaire. It queried the library user regarding his or her gcnr'er, 
student status, cKcupation, whether or not he or she requested staff assist- 
ance and was a registcTcnJ borrower, and general satisfaction with the 
librar>'s materials. However, Schlukbier wrote in 1978 that: "Any library 
that decides to change its library hours or staffing ratio based on data 
gathered from a six-quest!on 'user ticket' is asking for problems. "^^ 

OthcT performance measurement techniques which have been used, or at 
least proposed, include: (1) mc^asuring the time required to respond to 
inquiries in relation to the appropriateness of the information found; 
(2) tape recording telephone reference inierviews;^^ (3) observing patrons 
in their use of the library; (4) rnakrng tallies of reference and circulation 
activities; and (5) surveying library users, the entire community, and the 
library staff. 

Measuring the Quality of Library Seri'ice has a lengthy section devoted to 
"measuring techniques"^® Some of the techniques illustrated there 
jnc hide interviews, questionnaires, critical-inc idem tec hniques. consumer 
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panels, unobtrusnt' ufmiut* qiusiions, dian sunevs, luinstik' ( ounis, 
sdf-t'\aluations, documnii deli\n\ tests, operations leseanh, and niea- 
surement of mateiials a\ailal)dit\. 

With regaid to criteria for selecting and using appropiiate performance 
measurement techniques, A Planning Process for Pub Ik Libraries recom- 
mended that all objectives must first be measurable and then suggested 
certain criteria as an aid in designing methods of measurement. The 
authors indiraitd that the\ were most i.iterested in methods that were 
flexible, fairh simple, applicable for both one-time use and periodic 
evaluations, and pieferabl> alieady tested. Srhlukbier contended that: 
"Both quantitative and qualitati\e measuring techniques are required to 
concreieU express a iibran's performance."^^ With regard to quantitative 
measures, Howard and Norman noted that statistics must be anaU/ed 
regularly and must be used as an integral part of the decision-making 
process in order to beeffecti\e.^^^ Mar> Jo Lynch, in discussing the original 
DePiospo study, noted the desirability ofde\elopingc riteria which appeal 
descriptive of the effectiveness of a public librap program. And finally , 
Orr provided several desiderata foi measures of his "goodness of library 
services" concept. These were: (1) approi)riateness of the measure, 
(2) inforniativeness, (3) \alidit\, (4)reproduc ibilit\, (5) comparability, and 
(6) praciicalii\ .^^"^ 

On the surface, the \ariet\ of criteria for effective performance measures 
seems to he almost as great as the \ariet\ of measures themsehes. On the 
other hand, a closer examination of the criteria just discussed seems to 
indicate considerable o\ei lap in teims of what the criteria suggest is desired 
of perfoi mance measures. The major requirements of the measures appar- 
ently can be summarized as reliabilit\, \alidit\, and utility 

Possible Benefits 

But what of the benefits to be gained from measuiing the performance of a 
library? Why are more and more librarians, as well as other public service 
personnel, interested in pe rformance measures? The term that has surfaced 
most frequently in connection with measuring performance has been 
output. It has been pointed out by many that for evaluation purposes it is 
not adequate merely to measure input or the resources such as books, 
personnel, and equipment that are funneled into a library Rather, in order 
to obtain a meaningful e valuation of a library 's resources and services, it is 
necessary to evaluate or measure how well the library performs with these 
resources. Or in other words, how effectively does the library serve its 
community? 
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Just as ihviv IS no (onscnsus on what adiMiics cui Ix* txalualed uiih 
perfornian(f measures noi on what lt(hni(jues and tools (an Ix* used to 
nifasuie pcrfoinianc t\ iheitisa hukof agrtrmt*nio\fi potent lal benefits of 
perfor:nan(e measurement. Yet there is little disagreement with those who 
(ontend that performance measures do ha\e mu( h to offer the libiary 
adminisfrator. DeProspo, for example, wrote* "Aimed withsu(h informa- 
tion [performance measures], the administrator should Ix* able to make 
more judicious use of time and mateiialsand ha\ea fa( tual basis on wliuh 
to plan and make budget allocations."*^^ 

Howard and Norman, in their article on measuring [)ubli( library perfor- 
man( e, (les( rilxd their Q)mplete Service Statistics (CSS) as an in\ entory of 
all output reflecting user (ontact such as lending services, facilities ser- 
vices, information ser\i(es, production services, and staff services. They 
then identified some of the benefits that they expected to reali/e from 
(one(ting CSS, oi output measures, as follow a* 

1. Decision-making can now be based not onl\ on past exjXTiente, intui- 
tion, and guesswork but also on accumulated performance measures. 

2. Complete Service Statistics (an provide data for cost-benefit analysis. 

3. Performance-type data can reveal trends, changes, and directions in the 
system or in the community; obje( tives can be reviewed in light of any 
changes, and appropriate administrative action can then be taken. 

1. Accountability to the funding source and (he (oinmunitN at large (an be 

improved by such data. 
5. Having outpuit-tyix* information tends to improve the administrator's 

abilit> to predict future trends, needs, et(.*°^ 

Schrader expanded on the lx*nefits to be gained by de( isjon-makers in 
pointing out that they should: (1) have more quantitative knowledge of 
librar\ use and useis at their disposal; (2) be in a better position totompare 
their library with others; (3) be Ixrter able to interpret their library's 
perfoimance in terms of (juantified library obje(ti\es; (1) Uv better 
ecjuipped to develop an effe( tive public leldtions program; (5) ha\e avail- 
ai)le the tvpe of data often needed for "political purposes"; (6) be more 
effe( tive in designing librar> instru( tion aids sudi as signage; (7) Ik* able to 
impnne their a( quisitions dec isions and timing In making seasonal com- 
parisons; (8) have a greater ( han(e of s( hedulmgecjuipment repairs, et( . at 
times when tbe\ should (ause the least disruption in ser\i(es; (9) Ix* in a 
more insightful position for de\ eloping new libiary servK es; ( 1 0) be able to 
s( hedule staff most effec livel>; and (11) ha\ea\ ailable libraiy peiforman(e 
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measuifs whidi to sonu* t\u*ni should leflec i libicuv mciiiai^enu'ii? pt*rfor- 
maiKf.^^^ Other reasons j^iwn for mtMsuiiiit* output oi pt i loimam t* ha\e 
related to* 

1 gauging the effec ti\eness of .dteinati\e strategies, 

2, monitoring the consequences of vaning the allocation of lesouues, 

3. charting progress towaid the achie\ement of object i\es, 
\. assessing needs not being met, 

5 providing information whuh leads to better use of input, 

6. prox'idmg the rationale for changes, 

7. identif\ing social benefits that nia\ accrue, and 

8. analyzing the impart of priblu issues 

OddK enough, none of the juste i ted lists directly referied to the determina- 
tion of user satisfaction as an important benefit to be gained from measur- 
ing performance. Perhaps it v as considered to be implicit in many of the 
sjx^cific benefits identified, ho\ve\er, for Schrader stated that: "From a 
(Oiueptual standpoint the new methodology consideis user satisfac tion to 
Ix* both the ultmiate test of libiary effectiveness and, hence, ihe main 
predic lor of the extent of futuie libiary use."'°^ Zwei/ig and Dervin, c iting 
an earliei work by Paisley and Parkei, argued that user satisfac tion is an 
miportani criterion for the evaluation of a s\stem.*^ 

Limitations 

Yet, in spue of the fact that peiformance evaluaticjns often consider user 
satisfaction, and in spite of the fact that DeProspo felt justihed in saying 
that performance measurement "comes much closer than present statisti- 
cal reporting systems to prcniding user-oriented indicators, some crit- 
ics belie\e that many, if not most, performance measures have not placed 
enough ( mphasis on the user but rather continue to emphasize the mea- 
surement of input and or level of ac tivity. Powell, for example, following 
the Illinois test of the original Rutgeis manual, wrote that, "the perfor- 
mance measures manual.. tfX)ka rather traditional approach to the evalua- 
tion of public library services. Goldhor later wrote, "hi /egard to 
content, one can only be impressed by the richness of these measures in 
comparison with the traditional counts of total c irculation, attendance at 
hbrary programs, number of registered borrowers, ^'tc "^^^ "At the same 
time one can only hope that there will be explorations of new and different 
measures, particulaily in regaid to user satisfaction 

In fac t, the ability to measure usei satisfac tion acc urately continues to be 
elusive. In 1977, the technicjues presented in Performame Measwes for 
Public Libraries were used in the evaluaiicm of the public library of 
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Brampton, Oiuaiio. S(hra(U*i, in discussing this appliiation of DePros- 
po's perfoimancf measures, indKatal that sonu meaningful resuhs were 
a(hieved, but he also noted that the performanre nuasures basicalh mea- 
sured use and only indirec th measured user satisfaction. Or stated differ- 
ently, most of the performance measures "des( rihe and quantify the usage 
of materials and services In patrons, and it is from these indicators of usage 
that we can make inferences (rautioush) about overall user attitudes. "^^^ 

VVh\ has it proven so difficult to measure user satisfaction? According to 
E\ ms and others, there are certain problems with employing user satisfac- 
tion criteria to measure library performance. Among these pioblems is the 
fact that a strong subjective element is always present when one asks a 
library user to judge his or her satisfac tion with librar\ services. They also 
pointed out that measuring user satisfaction recjuires extensive testing of 
measures, training of personnel, etc. In addition, we have not yet deter- 
mined what we actually mean by "rek vant" with regard to the patron's use 
of the library. And we are still plagued by the old problem which we so 
often have to deal with in survey research— low response rates. Yet they 
continued to argue that, "user sa'isfaction must be considered one of the 
primary measures of library effectiveness.""* 

Researchers also continue to l)e faced with a lac k of knowledge regarding 
the variables that affect and indicate user satisfac tion. For example. Crane 
found in his study, and others, that the larger the 1 ibrar\ , the lower the level 
of user satisfaction. He hypothesized that users probably expec t more of a 
large library than they do of a small one."® Other research similarly has 
concluded that library users tend to be more satisfied with smaller c ollec - 
tions, as opposed to \ery large ones, probably because ol lower expec ta- 
tions and because the> find smaller collections easier to work with 

D'Elia and Walsh, in a 1983 artic le,"^ concluded that user satisfac tion is 
potentially useful for e\aluating the performances of services within a 
library but is not val id for c omparing libraries unless demograph ic c harac - 
leristics of the users can becontiolled. In a report of a follow-up study, they 
noted that changes in the collections and services in a library may not be 
p<*rc cived by patrons and the use of data c ollec ted from patrons for assess- 
ing the performanc e of libraries may be of cjuestionahle value. On the other 
hand, as the authors concluded, it could be that "we have not been asking 
the right questions, or user behavior is so idiosyric ratic that there very well 
may not be a parsimonious exi)lanati()n.""^ 

D'Klia, in a 1985 issuv o{ Puhlu Libraries, reported "that the data obtained 
from the materials a\ailability surveys of Output Measures for Public 
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Libraries an* apparently useless iiidiiatois of libiarv peiformaiKe."^^^ He 
based this ( one liision, in part, on his finding that none of thepation sear(h 
sua ess rates and none of the hll rates was signifuantlyc on elated with aiiv 
of the available per capita measures of libraiy resources for the St. Paul 
Public Library system — his test site. On the other hand, in an artic le in the 
same journal, \'an House stated that Output Measures jor Public Libraries 
is a useful tool for library management. **It })rovides librar> managers with 
measures and methods that, while imperfett, are practical and useful."'^^ 
In another artic le published the next year she concluded "The measures 
defined in OMPL that are l)eing used do seem to be valid measures of 
library perform ance."^^^ 

As was indicated earlier, a measure which has been used for some time to 
evaluate library performance has been library use. Yet Evans and others 
contended that "the units measured have not been ve ry prec ise or meaning- 
,,122 yj^^.y ^i^i^f} (hat theteare problems in employing "use" criteria in 
that: (I) they fail to distinguish between significant and insignificant use, 
(2) they seldom measure in-house use, (3) they are susceptible to radical 
variations, and (4) they fail to reflect the needs of potential users.^^^ 

Another problem with relying on use as a performance mc isure is related 
to the fact that use often is measured m terms of \olume only. "Measures 
which tell us that fewer people are using the library seivice but which do 
not indicate a need for such thingsas more books, different books or more 
libraries, aie of little use to the dec ision-maker."^^^ Or in other words, 
performance measu'^es too often do not piovide the kind of information 
needed to evaluate and improve services. Grayson and Wingate, for exam- 
ple, concluded tha* "As a measure of effectiveness, a gross circulation 
figure gives little indication of how well the 'information* function is 
being performed ' (And the same holds tiue for the recreational and 
educationa> functions.) 

Other limitations in employhig use statistics as performance measures 
have included the fact that too little information is gathered on library use 
in relation to the full potential and on library use which lies outside the 
primary service area, hi addition, use data often ha\e not been broken 
down into meaningful categories.^^^ In a paper presented at the 1985 I FLA 
Conference, OX^onnor went so far as to sa\ that since output measures are 
beyond library control, their use as a measure of library performance may 
be questioned. Yet he also stated that output measures result fiom 
characteristics of the user population (rather than lil)rar\ ac tivity)ancl that 
library output is influenced more b\ user c harac leristic s than library input 
Once again, a theme that seems to appeal is that if output measures arc 
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going tobt" \alid indiraiois of libran peifoimaiKr, ihe\ must adcqii a h 
imoiporateapi^ropriaK" user data. Thai issue will he addicssed in iht* next 
se( tion. 



PERFORMANCE MEASURES BASED ON I SER SI INDIES 

Befoie dealing with the relaiionship between iisei data and output mea- 
sures, the question of what we want performance induators to measure 
should be resoUed. hi other words, what concept best repiesentsa libiai\ 's 
performance? The answer may l)e "libiary effectiveness." As has Ijeen 
.stated in the literature, it may well be that "the final output of a library 
system is the effectiveness of the service/'^^® But what is meant by "libraiy 
effectiveness?" Redfein stated that: "Effectiveness may be defined as the 
extent to which a service can be said to meet the needs of the communitx , 
that is, both expressed and unexpiessed needs th .t lelate to librarv 
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purpose. 

The DuMonts contended that. "Librarians have yet to arrive at a deai 
meaning for the phrase libraiy effectiveness.* Although many library 
lesearchers write of evaluation and performance measures, they generally 
don't equate such concepts with a discussion of librar\ effecti\eness...."^^^ 
They said that it would be more meaningful to define library effectiveness 
as being related to the ac hievement of librarv goals whu h may "take the 
form of useful outputs which are consumed b\ those outside of the librarv 
system. 

More specifically, the authors considered libiar\ effec ti\eness to include 
consideration of- (1) goal achievement (admittedly difficult to measure), 
(2) efficiency, (3) user satisfaction, (1) jx^rsonnel input, and (5) s\stem 
goals. They pointed out that Rosenlx^rg, in 1969, argued "that the only 
criterion of effecti\eness is value received.' '^^^ The\ summarised by 
stating that 'library effec tiveness can Ix* \iewed as the successful interac - 
tion between the library and its environment. "^^^ In a 1981 Otcasional 
Papers, the DuMonts discussed librar\ effec tiveness and goals and how the 
two can be related. ^^'^ 

Assuming that there is a reasonable ( onsensus regarding the definition of 
library effectiveness, at least one more crurial question remains. V\K)V\ 
what source of information should the librarian base hisor her determina- 
tion of library effectiveness? It is being argued here tha: at least one logical 
source of this sort of information is the user study In other words, user 
studies can provide the researcher with the kind of output or performance 
measures that h;* or she needs to evaluate libraiy effectiveness as defined 
earlier. 
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In ia(i, tlicic is (onsidcrdhle support in the litnalurc tot iitih/ing iisci 
stiulus foi |xif()iman((' measures a. id, more spe(ifi(all\, for e\aluaiini* 
lilHtUx elfe( tneness. As Beeler slated "If we want to know how well .li- 
braries are ser\ini; the |K)pulati()n ue must turn to users of the servue for 
answers, he, iing in niind the 'nirposes intended hy the services "'^^Oras 
(lien and Ilernon wrote: "Ial)rar\ effec ti\ eness nnist he \iewed within the 
(onJ^'xt ol peoph s inlorrnali' n needs, the straiei^ies h\ wh!(h they gather 
inforniaiion, and the role oi source pro\iders in suppl\ing informa- 
llon/•'^^Sc huh/ went so far as to iie that. ' There is ( ()n>rderal)le agree- 
ment among in\estigaiors that ..ny perforrnance measure worthy of the 
name is user-oriented if it measures a service from the client's 

,,137 

v.ewjK)'ni. 

l^uuaster, (iting Tauln'r and Stephens, noted *i "the basic aims of a 
sur\e\ should Ix' e\aluaiion of the effectiveness of (he services provided, 
deterrr nation of the extent to which user needs are satisfied, and identifi- 
cation of wa\s in which ser\ice might Ix' impro\ed."'^^ Busha and Haner 
argued for the relationship Ixiween measuring librar\ effcTiiveness and 
gathering user related rnformaiion when the\ staled: "Among the ques- 
tions that librai pans h.i\e a distinct obligation to answer are thosv relating 
to librar\ effec tiveness, inc ludrng sue h fac tors as... the use and non-use of 
library materia I sand ser\ices ..thedegree of awareness about library collec- 
tions and serxices among c lientele or poteniial clientele of libraries. ..and 
user satisfaction or dissatisfac ticm with libraries. "'^^ Or, as they later 
stated, "librarians should \x concerned with the performance of their 
institutrcms.**'^^ Rodwell wrote: "There is a general recognition that mea- 
sures of effectiveness should adequately leflect the satisfaction of user 
needs. Similar emphasis on the importance of measuring user satisfac- 
tion in order to determine the library's contribution to its c ommunity can 
found in Hc>adley and Clark's work on quantitative methods. 

All twelve of the measures in Output Measures for Public Libraries reflec t 
user activities to one degree or another. Kantor's four performance 
measures for academic and research libraries are patron oriented. Cro- 
nin's model for assessing public services in academic and research libraries 
suggests thai four factors should be c onsidered when setting a standard for 
cjuair of seiMce.'^^ One of those factors is 'user expectations of the 
service." McClure. in a consideration of performance measures for corpo- 
rate libraries, stated: "Performance measurement involves the establish- 
ment of library C)f)jectives based on user rx'eds, the expression of these 
objectives \n quantifiable units, the "asurement A the units, and the 
assessment of library performance \is a-vis its stated objectives. "'^^ In a 
re\rew of research on reference service effec tiveness, Powell noted that tfie 
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majol rmphaMs had Iktii on ()ir[)iit intasuu s and iUm i\\uiv a ivw oiihv 
incasiiu's had to do »vilh iht* saiisiadion oi usns' nmls ^'^^ 



A Planning Process fo) PubUi Libraries siaU'd thai [U'lformancf mrasuics 
(an Ik* (attgori/ed in a nuiTilx'i of \va\s, ni( hiding l)\ ihr p()j)ulation 
groups (onsukud Schhikbiti also stussrd the importancr of ha\ nig 
adequate informaiion about the hhiau \ rommuniiN when he staled that 
(he "uhniiaie e>ahialion" of an\ hbran must be based on the use being 
made b\ the ( omnium t\ , and aigued thai pei format (r ( annot be measure(l 
wilhout knowledge of the ( omnunnlv ^'^^ S( hlukbiei (onrluded. "We as 
professionals ha\eaii obligation to i)rodure(onstru( tive lesearc h that will 
give u^ \iial mfoimation (OiKeining the needs, desires, .uid indiMdual 
(haia( teristK s of useis and noii-useis. PeifoimaiKe measurement is the 
onl\ veliK le to ade(juately exploit' that iiegletted area of hbrarianship, 
while^)ro\ iding a lelativeU simple design for effec lively managing librar- 
ies.** Daniel Gore, writing about his use of DePiospo's performance 
measuies, said thai "theonU iiustwortin measure of Holduigs and Avail- 
abilii\ Rates tue those whi( h areapphed dire( ily to the a( lual users of any 
gi\en lil)iar>, whether it be publn , academu, oi spec lal."*^* And finally, 
Lowell Martin su((in(tl\ stated: "Usei information is a ke\ (omponentin 
nieasureineni and e\aluaiion '**^^ 

Others ha\e (oniended that in ordei to c\aluaie more fully the ultimate 
output or effe( ti\eness of a public library , we must learn more about how 
and why library users use the libiary. Zwei/ig and Dervin, for example, 
stated that "more might be ganu 1 b> mo\ ingfrom usei studies to studies of 
(he uses to which public libraries are pui.'**^ The DuMonts wrote that 
hbrai.jiis need to measure uses, as well as users Ford, in his work on 
user studies, wrote: 'Key f utoisin studies of information transfer are thus 
the pui pose for which information is sought, and the use whic h is made of 
it. These fac tors aie often neglected, or tieated only superficialh in many 
user studies.'**^ Totterdell argued that final output, oi the effec tiveness of 
a libiary service, should be measuied in teiins of use, inc hiding total use of 
the library and (he degree of use b\ different age gioups. social classes, 
occupational groups, and different geogiapliical areas within the total 
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ser\ ice aiea. 

Whethei the emphasis is on the user and his oi her characteiistics or on 
how the user uses the library, theie can be little doubt that the professional 
literature' e\ idences 'substantial sU[)port among lil)rarians foi relating user 
information to the evaluation of libiary effectiveness, hi addition, there is 
considerable endorsement for \iewing library effectiveness as perhaps the 
most important element of libiary peifoimance. But does the literature 
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pioMck' mii(h e\itk*ntt' thai lil)rarians praduf what ihcv puadi? lUw 
libiaiians in lac t aitcmpicd lo mt asiirt* (lit* pt rlormaiue oi rik'i incncNsof 
then libraiKN l)\ mikKiii^ then IrbrarKs' users? 



Speaking of libiai\ effe( lueness ni geneial, the DuMonis wiote that ' lehi- 
tiveK httk" is known about the elfe( tneness of man\ hbiai\ progiams" 
and that there is a "kuk of obje(ti\e empnical e\ahiati()ns*' of libtary 
servKes'^^ With regard to the use of usei studies in measuring perfor- 
man(e. Knightlv anah/ed the 1977 and 1978 annual leports of 62 sele( ted 
libiaiies of \aiious tyiK's and found onl> 1.1% ol the performance criteria 
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being used were user studies. DePiospo lepoited in 1976 that there was 
almost no literatme discussing the wa>s in w huh ( ommunit> anahsisand 
the librarN measurement process can or should be brought together He 
pointed out that* "The situation is furthei (ompluated b> the typical 
pi obi em, that the library's existing measurement process has generated an 
information base laigel\ unsuitable for (ompaiing results with existing 
coinmunitN information data bases. 

There have been some libraiy e\aluatior studies that ha\e taken into 
account the libiaiN user. However, "most ol oiircurient library effective- 
ness studies, particularly those fiom acackmic libraries, ecjuate library 
effectiveness with user satisfac tion.* '^ (As was noted earliei, this limita- 
tion was reflec ted in a criticism of the original DeProspo studv. in that his 
major direct measurement of users was lelated to usei satisfaction ) 

As was reported earlier, the Rutgers manual on public library performance 
measures was tested by the l^niversitN of Illinois Librar> Research C^entei. 
Following the Illinois stuck, the Researc h Center sent a questionnaire to a 
sample of the participating libraiians asking them about the antic i pa ted 
usefulness of the data obtained as a result of their ha\in^ conducted a 
performance measures stud>. Tser infoimation was the response most 
frequently c ited as the type of ckita from w hic h the\ expected to beneht 
As a lesult, the Librai\ Rest uc h Center lec ommenck'd expanding the user 
ticket (questionnaire) and distributing longer usei cjuestionnaires to a 
larger sample of pat ions 

In a study leported in 1980, Detweiler in\estigated the relationship 
between library effec tivenessand thea\ailabilit\ of materials sought by the 
libiary pa tron/^^ Howard and Norman designed their Complete Service 
Statistics to inventory all library output reflecting user contac t including: 
lending services, facilities services, information services, production ser- 
vices, and staff sen ices/^'' Jones stated that: "It is often con\enient to 
attempt to assess the adequacy of a library service in relation to the 
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population scr\ ('(!," ihout^h he seem 'dlolakca basualK uipui-ouenlcd 
approach in his siud>. 



Ford, reporimg on his surve> of user studies, induated that se\eial fa( tors 
affecting user information needs had l)een identified Ihese included, 
among others: (1) the backgiound, niotuation, professional orientation, 
etc. of the user; (2) the social, political, economic, and other systems that 
affect the user and his work; (3) the uses to which information is put; and 
(4) the consecjuences of information use — foi example, the prochu tivity oi 
the user.^^ 

Data Collection 

But what of the techniques used toapply user information to performance 
measurement in these studies and others? What are the vaiiables deemed 
important enough to measure? A surve\ of the literature identified a 
considerable number of techniques and \aiiahles, but the distinction 
between the variable or characteristics to be measurc^d and the measure- 
ment tec hnique was not always apparent Howevei , what follows is at least 
a partial listing of the terms that appear to have been used to identify or 
.epresent user-oriented vaiiables worth considering in performance mea- 
surement. Following that is a list of techniques pioposed or used to 
niv^asure one or more perfoimance variables Usei \anables related to 
library performance include: 

1. user needs, 

2. user use, 

3. user satisfaction, 

4. patron's expectations, 

5. efficiency, 

6. process, 

7. demographic -type data, 

8. community satisfaction with its hbiai\ serMces, 

9. in formation -seeking bahaMoi, 

10. purposes for seeking information, 

1 1. user's personality, 

12. user's interests, 

13. user's attitudes, 

14. user's "total-hfe situation," 

15. user demands. 

16. user "fac tors," and 

17. citi/en awareness of library services. 
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rsri-it'hitcd lt*( hriKjiU's ii>t*(l !() intMsiirt* lihraiN ptifoimcUKf iiKluck' 

1. niathrmatKal nKxkK (Ifscribin^ lihrarx iist\ 

2. prow goals— f g., substituting t vposiiu' liiiK* loi \ahu* ol dcx umrnts to 
u sm, 

3 s\sirms appioa( h, 
}. (ommunitN anahsis. 
f). lihrarv rfs|X)nsf tiint', 

6. do(umnit (k li\( r\, 

7. proMsion of ( itations, 

8. total Iil)rar\ (ontad tiiiK* pt'i pottniial usn, 

9. "itt*ni-usf-da\,'' 

10 fxlnii of rfackT sfll-sfi\ u t*. 

11. cost htiK'fit ratio, 

12 lihraiv tffort (input), 

13. library performance (output), 

H, st*ai(h su(tfss rate, 

15. total Iibrar\ use, 

16. penenf ige of total population being ser\ed h) the library, 

17. penenttige naterials used a((oi(ling to t\pe of user, 

18 ratio of a given ser\i(e to the total niimlxi of library iiseis, 

19 ratio of the numbei of documents (iKulated to types of users, 

20. ratio of total use to total holdings, 

21. user access, 

22. return visits, 
23 use log, and 

21. program attendarrce log 

As is evident from a reading of the two pi'/cedmg lists, not all of the items 
mentioned appear to l)e based entirely o;i the library user As was noted 
earlier, many "peiformaiKe" measures are in fact more concerned with 
libraiy activities than with librarv users (or measures). Yet one measure 
hat rs surely user -oriented, user satisfaction, is probably cited more fre- 
']uentl\ than an\ other measure as an important, if not essential, inclrc ator 
of libiar\ {)erfor inane e, 

I;uic aster, for exampk', identified several important fac tors that affect 
library performance and indicated that they should be judged in terms of 
user satisfaction.^^ Park<.r stated that in order to measure library perfor- 
mance one needs inlormaticm about, among others, user satisfac tion.^^^ 
The PLA s recent work charged the library pkinnrn^j; committee with 
prc)\ iding periodic information on c hanges in users' satisfac tion with the 
library. The PLA's work on output mt^asurement also suggested the 
importance of user satisfac tion.^^^ Burns recommended a set of rnckxes of 
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effectiveness to measure the relationships between levels of libiar\ perfor- 
mance, resouire allocation, and user satisfaction. E\ans and others 
reviewed the criteria that ha\e lK*en used to measure library effectiveness, 
and they included, among others: general user satisfaction and user satis- 
farticm with existing libiary ser\ices and materials/'^ 

Zweizig concluded that user satisfaction is the best measure available for 
measuring library use, but he did point out that it does not reveal how the 
library's resources actually help the user/'^ (This issue will be considered 
again at a later point.) White cautioned that such ter is as user needs and 
user demands are sometimes used interchangeably, but he noted that needs 
are more difficult todetermine than aredemands.^'^The DuMonts, in their 
work on rr •a'^uring library effectiveness, contended that it is best to takean 
integrate',' ";jproach to evaluating I brary effectiveness. They suggested 
considering elements such as the employee, the library as an organi • uion, 
and the total environment. They also emphasized the importance of unhz- 
ing measurement that is relevant or meaningful. Or in other words, 
regardless of the variables or techniques selected for measuring library 
performance, they should be approrpriate for the task and capable of 
providing useful data and ultimately subst;*ntive benefits. 

Possible Benefits 

Having discussed already the many benefits potentially obtainable from 
utilizing performance measures and user studies, it probably is not neces- 
sary to reiterate the benefits to be gained from emplo>ing them separately. 
The major point to be made here is that by basing performance measures 
on data gathered in user studies, the librarian is more likely to have at hisor 
her disposal truly meaningful data. Rather than having a* ailable data that 
merely descnbe library activities or do nothing more than characterize 
library users, the librarian should be able toaccjuirc information about 
how^ library users and library services interact. Or in short, the application 
of user studies to performance measures should result in a more accurate 
assessment of a library's effectiveness. 

There has been some discussion of the benefits to be gained from employ- 
ing user studies as performance measures and oi measuresof effectiveness. 
The DuMonts. having pointed out that user satisfac tion is one element of 
the definition of library effectiveness, stated that "the primary purpose of 
measurement procedures is to obtain information about the library in 
order to provide decision makers with a clearer understanding of what the 
library is doing. An accurate and objective evaluation of the effectiveness of 
the library can then beattempted. "^'^ Similarly, Lynch wrote: 'Theresults 
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of measurement (an Ix* used to exaluaie (he peifoimaiKe of a libiai\ and 
(hereby de(ermine **'he(her or not it iseffec ti\ e. '* S( hlukbier argued (hac 

PerformaiKT measurement is a rescarc h lool < apable of rlarif\ uig f)bj('( - 
lu es of library servues, and mduaiuig duedions foi giow(h and i('(h- 
niqucs for mcreasmg the effuiencv of iheu implcmeniauon The 
uluma(ee\aIuauon of 'iny Iibrar\ must be made ui icinisof theusebeuis 
made by ihe r{)mmunn>. Therefore, fmdmgoui how a sped fir library or 
library s>sieni is meeung ihe needs of i(S(ommunn> will re\eal areas in 
which efficienc) can be impro\ed and library growih enhanced 

1 he authors of A Planning Process for Public Libraries emphasized that* 
"Planning library ser\ices for a community requires, first of all, an under- 
standing of thai rommunit>: its environment, iis population, their infor- 
mation needs, and the sources available to meet those needs."^'^ Lancaster 
stated that a well -conducted library sur\ey should attempt to assess the 
degree to which the library services meet the needs of ihe community 
served. Doing so can provide a useful indication of how satisfied the users 
are with the ser\ ices provided. Schrader also stressed the importance of 
assessing user satisfaction. He contended that the user iscomemed about 
how accessible the library and its services are; oi in other words, how much 
time and effort the user will have to expend in order to satisfy his or her 
information needs.^^ As DeProspo has stated: "A better educated and more 
sophisticated public is less willing than ever to accept the need for com- 
munity services on faith alone. Increasingly th^ public is demanding proof 
of the effectiveness of various programs. "^^^ 

In summary, performance measures based on user studies can provide 
mamgement with the kind of information it needs lodefine, quantify, and 
mea^iure a library's success in accomplishing its service goals. "They 
enable the institution to tell how, why, where, and when it is successful and 
to express this success in meaningful, i.e., quantifiable, terms for compari- 
son with similar institutions or systems, "^^^ and just as importantly, in 
terms ihai are based on the library user as the unit of measure For as Burns 
stated, "public service agencies such as libraries need continuous feedback 
from their use* s lest they lose touch with the realities of their existence."^®^ 

Limititions 

Having pointed out some of the benefits to be reali/cd (^f'^m employing 
user-^rienied performance measures for the e\aluaiion of a library, it is 
importanr to recogni/e that such iui approac h also has cei tain limitations. 
Some of these limitations are applicable to measurement procedures in 
general, others are specifically related to user-based measurements. 
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Dt'Prospo, foi t'\amf)k\ (aiiiioned ihat "noi all lil)iar\ a( ti\ nivs aiesubja t 
to ieasonal)lt* quaniifi(aiion or ol)jecti\e nKasiircmcni "^^^ 

Thv DiiAMonts staled that: ' 'I hv study of library v{{v{ iiwrn ss is made niore 
romplex bv the ambiguous, relati\isti( character of the ernironment of 
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which the hbrar\ is a part." rhe\ later itemi/ed some of the measure- 
ment problems that tend to exist, and these uk hided. 

1 the frequent reluctance of librarians to use certain measuiement 
technicjues; 

2. the fart that man\ work load indicators, such as the numbers of books 
cataloged, circulated, etc., sa\ nothing about effectiveness; 

3. the not necessarily true assumption that financial support is an ade- 
quate indicator of effec ti\eness; 

4. the limitations of ph\sical standards, 

5. the fact that cost-benefit analysis places too much reliance on value 
judgments; and 

6. in order toemplo> user studies to measure librar\ effec ti\eness, criteria 
must be used that accurately reflect the basic ser\ices of the library as 
they affect those who use or do not use them. ( The\ also reminded the 
reader that different groups have different goals and different criteria for 
:*valuating a library's effectiveness.) 

It was suggested earlier that among the most \alid criteria for measuring 
library effectiveness is patron use of library -supported information. But as 
Zwei^ig acknowledged, measures of how the produc ts of library service are 
used by patrons are very elusive, and the least under the control of the 
library. 

Another limitation is related to the need to study thenonuser,as well as the 
user — an assumption implicit in most of what has been stated thus far 
Unfortunately, it rse\en morediffic uL to survey the non user than the user. 
For example, it has been found that: **Tiiereis a high correlation between 
nonreturn of questionnaires and non-use of the library." 

It also has l)een indic ated here and elsewhere thai it is desirable r) investi- 
gate user needs as well as demands. But this too has pro\en dilficult io 
accomplish, and most user studres ha\e dealt with demands rather than 
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needs. 

User satisfaction, another important reflection of library performance 
discussed earlier, has likewise proven hard to determine. D'Elia stated: **To 
the extent that user satisfaction is a function of the library's ability to 
gratify th^* usct's needs or fulfill his expectations, user satisfaction is 



{K)U*n(iall\ iiww useful diaKiiosiu mtMsiireoi librarx iKifoimaiKc Tht- 
fxtent of this usffuliKss, houtAcr , is dt'iKridtni upon oni cil)ilu\ to i(kntil\ 
ihv dfifrminants < f istT satisfiu tion.^®^ llv wvm on to piopost- that iis^-i 
saiisiadion is a fiiiK tion of tht' iist r\ dt*mot»rai)hi( ( haia( trrisiK s, iht* 
\ arious Uses madt- of tht- lihrai \ h\ the iist r , and tht- nsfi 's v\ ahiation of iht* 
( hara( icrisiKs of tht- h!)iai\ iisfd. 

Biukland, in his im[)ortant work on lK)ok a\ailabihi\, (onchidfd that: 
" The natiiif of tht- it lationship hetwren iisfi lKha\ioui and Satisfaction 
l.v\v\ IS not \tt (k*ai. Ih- e\ni h\ pothtsi/ed that ustrs' demands for 
books wvrv afff( tt-d h\ then satisfac tion lf\els, w hu h in turn affec ttd their 
demands for books. 

A similar (oiKtin was expiessed b\ Stipak in hisanide on the potential 
misuse of ( ili/en satisfaction as a peiformance measure of uiban servues. 
He advised that expressed satisfac tion ma\ not reflect a( tual service perfor- 
niame, and that indicatois must be Imked to specific ser\ires. He also 
cautioned against reUing too hea\il\ on the conceptual and statistical 
anahsis of esseniiall\ subjec ti\e indicators sue h as user satisfaction. In 
short, user satisfac tion alone ma\ not beasatisfac loiyc ri tenon for measur- 
ing library peiformance. And no doubt this limitation applies toother 
performance measures as weii 

In light of the \aiic)us strengths and weaknesses of user-oriented perfor- 
mance measuies, and consick ring the support at least indirectly e\idenced 
for them in the liteiature. what does the future appear to hold for develop- 
ing performance measures based on user studies? Is the 'situation or out- 
look an\ better now than it was in 1976 when DeProspo wrote* 'While 
undoubiedh the most desirable mc^asurenieni process is one which pro- 
duces data on outcomes, that is, data whic h prove that needs are met and 
lxha\ioi fhanged as a lesult, we remain unable to execute such a pro- 
cess." Possd)l\ not, but there at least have Ix'en suggestions forthcoming 
as to how the effec ti\eness of using usei studies to evaluate services can be 
mc reased. 

Stipak recommended »hat we not reh too heavily on survey items asking 
persons how satisfied they aie with local services or asking them loac lually 
e\ ablate partic ular ser\ ic es. Rather we should c one entrate more on asking 
specific, objective cjuesiions probing c iti/ens' actual use of librarv ser- 

193 ' 

vices. Along these same lines, I^uic aster noted that McCarthy and 
Howder "believe that the general survey will be replaced b\ more spec ial- 
i/ed studies and that more sophistic atedanaUses and evaluations. ..ma\ be 
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requirc^d. 



Lubans (omt nded thai librarians si.oiild iruf'sngaic fadors such as user 
siu(ess rn using spt( ifi( librar\ srr\icessuth as thecard (atalog, referernt' 
sm ni\ and !)ooks in ihv stac ks, ultimate user benefits reali/ed from obtain- 
ing information from the libiar\; the social and economic impacts of 
librar\ use and nonuse: iind users' needs for knowledge on how to use the 
hbrar\.^^^ Zwei/ig argued for the importance of exploring further the 
c]uality or type of librarv use.'^ 

The importance of continuing lo consider both user needs and demands 
and to distinguish between the t\NO was stressed b\ the DuMonts The> 
wrote: 

Individual patrons have \m specific mfoimauon nmis which the 
!ibrar\ c an fulfill, a small proportion of w hic h arc \ crbali/ed as demands 
which ihehbrar\ is cxpcc ted to fill if u is to be judged effcc luc The first 
step m achieving effectiveness is aciuaih idenuf\ing or defining what 
these information needs and demands are. Fulfilling user demands is an 
iniermediiiie step in the attempt to be effectue. Fulfilling needs is the 
uliimaie goal 

In c oncliision, Zweizigand Der\!n reminded the reader that, **we must give 
attention to the 'uses' that will be made of the *user' studies by public 
librarians. The important question i» whether the research helps.. .in eval- 
uating effectiveness. "^^®Zweizig noted that in 1978, the National Center for 
Education Statistics (NCES) assembled a panel to make recommendations 
as to how public library statistics might be made more user-oriented. The 
panel was to recommend areas for study regarding public library users and 
nonusers and servic es. Major questions resulting from the panel's delibera- 
tions related to the current effec tiveness oi public libraries and focused on 
(I) the function of the library for the community, (2) the portion of the 
community being served adecjuately, (3) the effects of particular library 
programs on users, and (1) the development of techniques for demonstrat- 
ing that library use had helped users to improve their lifestyles or had 
benefited them in oi /r ways. The panel concluded by proposing further 
study in the areas of user studies, the social impact of libraries, library use 
models, service inventories, the impac t of administrative dec isions, and the 
application of public service sur\e\s and user nonuser siirxeys toadminis- 
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iraiive processes 



SUMMARY AND C:ONCLrSIO\S 

This paper has attempted to point out that if libraries ever hope to truly 
evaluate their services, they must employ valid procedures, or measures 
that in fac i measure what they are intended to measure. Or as Lancaster has 
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slated. "Onl\ b\ appKing appropriate rmasuremrni and evaluation tech- 
niques can a librar\ determine the ( irc umstances under which it performs 
well or less well and identih the causes of its failures with sufficient 
precision to allow correc ti\e ac tions to improve the overall level of perfor- 
mance and, presumably, to raise the level of usei satisfaction with the 
services provided. "^^ 

This paper has suggested that what libraiies should most be concerned 
with measuring is their ultimate produc t — performance or effectiveness — 
and that the best indicators of their level of performance are, or shouL be, 
based on user data such as satisfaction. These two points were made earlier 
by, among others, Armstrong when he wrote that: ' The ultimate point of 
measurement and evaluation in an institution is its product. Does it 
produce what it is designed to jjroduce, and does the product meet the 
requirements of the customers?" And later. "Of all the generally feasible 
prcxess measures, the one that comes closest to establishing a product is the 
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user survey. 

In short, there is a leal need for libraries to be accountable for the effective- 
ness of their services. And at least one possibly valid approach to evaluat- 
ing the effectiveness of library services involves user-oriented performance 
measures. Employing such an approach takes cogni/ance of the benefits of 
using performance measures and user studies sepaiaiely, integrates the two 
techniques, and, it is hoped, produces anevt n better method for evaluating 
the performance of libraries. 
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